The present invention relates to an automated method and system for processing digital radiological images, and more specifically, to a Divide and Conquer (DAC) method and system for the detection of abnormalities, like lung nodules, in radiological chest images using zone-based digital image processing and artificial neural network techniques.
Lung cancer, next to heart disease, is the second highest leading cause of death in the United States. Successful detection of early-stage cancer tumors is able to increase the cure rate. Detection and diagnosis of cancerous lung nodules in chest radiographs are among the most important and difficult tasks performed by radiologists. To date, diagnosis in x-ray chest radiographs is the most important diagnostic procedure for the detection of early-stage, clinically occult lung cancer. However, the radiographic miss rate for the detection of lung nodules is quite high. Observer error, which causes these lesions to be missed, may be due to the camouflaging effect of the surrounding anatomic background on the nodule of interest, or to the subjective and varying decision criteria used by radiologists. Under-reading of a radiograph may be due to many other reasons, such as lack of clinical data, focusing of attention on another abnormality by virtue of a specific clinical question, etc. However, most peripheral lung cancers are visible in retrospect on previous films. Thus, a need remains for an automated method and system for digital image processing of radiographic images to alert radiologists to the location of highly suspected abnormality areas (SAAs).
Early radiological detection of lung nodules can significantly improve the chance of survival of lung cancer patients. A system capable of locating the presence of nodules commonly obscured by overlying ribs, bronchi, blood vessels, and other normal anatomic structures on radiographs would greatly improve the detection process. The automated system and method of the present invention allow the reduction of false negative diagnoses, and hence lead to earlier detection of lung cancers with high accuracy.
Several computer-aided diagnosis or detection (CAD) techniques using digital image processing and artificial neural networks have been described in the open literature and in patents. Of particular relevance to the present invention are the following:
Michael F. McNitt-Gray, H. K. Huang, and James W. Sayre, xe2x80x9cFeature Selection in the Pattern Classification Problem of Digital Chest Radiograph Segmentationxe2x80x9d, IEEE Transactions on Medical Imaging, Vol. 14, No. 3, September 1995, describes a method for the segmentation of digital chest radiographs using feature selection in different anatomic class definitions. McNitt-Gray et al. apply stepwise discriminant analysis and neural network techniques to segment the chest image into five anatomic zones. The five anatomic classes are: (1) heart/subdiaphram/upper mediastinum; (2) lung; (3) axilla (shoulder); (4) base of head/neck; and (5) background (area outside the patient but within the radiation field). This method was developed for use in exposure equalization. Note that the segmentation method of McNitt-Gray et al. is based on well-known knowledge of the anatomic structure of the lung region. Additionally, the zone boundaries in the McNitt-Gray et al. paper are crisply delineated and do not overlap.
Ewa Pietka, xe2x80x9cLung Segmentation in Digital Radiographsxe2x80x9d, Journal of Digital Imaging, Vol. 7, No. 2 (May), 1994, uses a three-step algorithm involving histogram-dependent thresholding, gradient analysis, and smoothing to identify lung and non-lung regions. The method is developed for use in exposure equalization.
Jeff Duryea and John M. Boone, xe2x80x9cA fully automated algorithm for the segmentation of lung zones on Digital Chest Radiographic Imagesxe2x80x9d, Medical Physics, 22 (2), February, 1995, describes a multi-step edge-tracing algorithm to find the lung/non-lung borders and hence to identify the lung and non-lung regions. The method is not developed for CAD purposes. The method is developed for use in exposure equalization.
Samuel G. Armato III, Maryellen L. Giger, and Heber MacMahon, xe2x80x9cComputerized Detection of Abnormal Asymmetry in Digital Chest Radiographsxe2x80x9d, Medical Physics, 21 (2), November, 1994, describes an algorithm to detect abnormal asymmetry in digital chest radiographs using multi-stage gray-level thresholding. The purpose is to identify the left and right lungs and to detect large-scale abnormalities, like the asymmetry of the two lungs. The method is not developed for CAD of lung nodules.
Maria J. Carreira, Diego Cabello, and Antonio Mosquera, xe2x80x9cAutomatic Segmentation of Lung zones on Chest Radiographic Imagesxe2x80x9d, Computers and Biomedical Research 32, 1999, describes a method for automatic segmentation of lung zones in chest radiographic images. The purpose of the method is to use the lung zones as a first estimate of the area to search for lung nodules.
Neal F. Vittitoe, Rene Vargas-Voracek and Carey E. Floyd, Jr, xe2x80x9cIdentification of Lung regions in Chest Radiographs Using Markov Random Field Modelingxe2x80x9d, Medical Physics, 25 (6), June, 1998, presents an algorithm utilizing Markov Random Field modeling for identifying lung regions in a digitized chest radiograph. The purpose of the algorithm is to identify lung zone so that specific computer-aided diagnosis algorithms can be used to detect lung abnormalities including interstitial lung disease, lung nodules, and cardiomegaly. Note that the CAD method of Vittitoe et al. is limited to the identified lung zone and ignores the obscured lung regions, such as mediastinum, cardiac, and subdiaphragmatic areas.
Akira Hasegawa, Shih-Chung B. Lo, Jyh-Shyan Lin, Matthew T. Freedman, and Seong K. Mun, xe2x80x9cA Shift-Invariant Neural Network for the Lung Field Segmentation in Chest Radiographyxe2x80x9d, Journal of VLSI Signal Processing 18, 1998, describes a method of using a shift-invariant neural network to segment the chest image into lung and non-lung zones. A set of algorithms is used to refine the detected edge of the lung field. Hasegawa et al. do not further segment the lung zone into different zones. Though mentioning the potential usage of their result for the CAD applications, they discard all the pixels in the obscured areas in the lung zone. The paper suggests that CAD be applied to the non-obscured areas of the lung, while the obscured areas, such as heart, spine, and diaphragm, are excluded.
Osamu Tsujii, Matthew T. Freedman, and Seong K. Mun, xe2x80x9cAutomated Segmentation of Anatomic Regions in Chest Radiographs Using an Adaptive-sized Hybrid Neural Networkxe2x80x9d, Medical Physics, 25 (6), June 1998 (the article also appears in SPIE, Image Processing, Vol. 3034, 1997), describes a method of using image features to train an adaptive-sized hybrid neural network to segment the chest image into lung and non-lung zones.
U.S. Pat. No. 4,907,156 to Doi et al. describes a method for detecting and displaying abnormal anatomic regions existing in a digital X-ray image. A single projection digital X-ray image is processed to obtain signal-enhanced image data with a maximum signal-to-noise ratio (SNR) and is also processed to obtain signal-suppressed image data with a suppressed SNR. Then, difference image data are formed by subtraction of the signal-suppressed image data from the signal-enhanced image data to remove low-frequency structured anatomic background, which is basically the same in both the signal-suppressed and signal-enhanced image data. Once the structured background is removed, feature extraction is performed. For the detection of lung nodules, pixel thresholding is performed, followed by circularity and/or size testing of contiguous pixels surviving thresholding. Threshold levels are varied, and the effect of varying the threshold on circularity and size is used to detect nodules. Pixel thresholding and contiguous pixel area thresholding are performed for the detection of lung nodules. Clusters of suspected abnormality areas are then detected.
J. S. Lin, S. C. Lo, A. Hasegawa, M. T. Freedman, and S. K. Mun, xe2x80x9cReduction of False Positives in Lung Nodule Detection Using a Two-Level Neural Network Classificationxe2x80x9d, IEEE Transactions on Medical Imaging, Vol. 15, No. 2, April 1996, describes classification of nodules and false positives using a two-level convolution neural network consisting of a two-dimensional connection trained with a back-propagation learning algorithm. The two-level convolution neural network is trained based on all the candidates of the whole lung zone. Lin et al. focus on the classification of nodules and false positives (i.e., on reducing the number of false positives). Lin et al., without mentioning the segmenting of lung zone of the chest image, apply the search techniques to the entire chest image to identify suspected abnormality areas, for example, an area containing a nodule. The nodules and false positives initially detected in the whole lung are used to train the two-level convolution neural network. S. C. Lo, M. T. Freedman, J. S. Lin, and S. K. Mun, xe2x80x9cAutomatic Lung Nodule Detection Using Profile Matching and Back-Propagation Neural Network Techniquesxe2x80x9d, Journal of Digital Imaging, Vol. 6, No. 1, 1993. This article describes nodule detection methods using a fully connected neural network trained with a back-propagation learning algorithm and a two-dimensional sphere profile matching technique. Lo et al., without mentioning the segmenting of lung zone of the chest image, apply the techniques to the entire chest image to identify suspected abnormality areas. The approach used in Lo et al. article does not segment the lung zone in the chest image. The search and classification of false positives are not limited to the lung zone but are applied to the whole chest image.
J.-S. Lin, P. Ligomenides, S.-C. B. Lo, M. T. Freedman, S. K. Mun, xe2x80x9cA Hybrid Neural-Digital Computer Aided Diagnosis System for Lung Nodule Detection on Digitized Chest Radiographsxe2x80x9d, Proc. 1994 IEEE Seventh Symposium on Computer Based Medical Systems, pp. 207-212, describes a system for the detection and classification of cancerous lung nodules utilizing image processing and neural network techniques.
S. C. B. LO, S. L. A. Lou, J. S. Lin, M. T. Freedman, M. V. Chien, and S. K. Mun, xe2x80x9cArtificial Convolution Neural Network Techniques and Applications for Lung Nodule Detectionxe2x80x9d, IEEE Transactions on Medical Imaging, 1995, Vol. 14, No. 4, pp 711-718, describes a system for detection and classification of lung nodules using a convolution neural network. M. L. Giger, xe2x80x9cComputerized Scheme for the Detection of Pulmonary Nodulesxe2x80x9d, Image Processing VI, IEEE Engineering in Medicine and Biology Society, 11th Annual International Conference (1989), describes a computerized method to detect locations of lung nodules in digital chest images. The method is based on a difference-image approach and on various feature-extraction techniques, including a growth test, a slope test, and a profile test. The aim of the detection scheme is to direct the radiologist""s attention to locations in an image that may contain a pulmonary nodule, in order to improve the detection performance of the radiologist.
U.S. Pat. No. 5,463,548 to Asada et al. describes a system for computer-aided differential diagnosis of diseases and, in particular, computer-aided differential diagnosis using neural networks. A first design of the neural network distinguishes between a plurality of interstitial lung diseases on the basis of inputted clinical parameters and radiographic information. A second design distinguishes between malignant and benign mammographic cases based upon similar inputted clinical and radiographic information. The neural networks were first trained using a hypothetical database made up of hypothetical cases for each of the interstitial lung diseases and for malignant and benign cases. The performance of the neural network was evaluated using receiver operating characteristics (ROC) analysis. The decision performance of the neural network was compared to experienced radiologists and achieved a high performance comparable to that of the experienced radiologists. Asada""s method appears to be limited to the detection of lung diseases exclusive of lung cancer, which present different symptoms. Asada""s method also does not use a zone-based approach.
Y. S. P. Chiou, Y. M. F. Lure, and P. A. Ligomenides, xe2x80x9cNeural Network Image Analysis and Classification in Hybrid Lung Nodule Detection (HLND) Systemxe2x80x9d, Neural Networks for Processing III Proceedings of the 1993 IEEE-SP Workshop, pp. 517-526. The Chiou et al. article describes a Hybrid Lung Nodule Detection (HLND) system based on artificial neural network architectures, which is developed for improving diagnostic accuracy and speed of lung cancerous pulmonary radiology. The configuration of the HLND system includes the following processing phases: (1) pre-processing to enhance the figure-background contrast; (2) quick selection of nodule suspects based upon the most pertinent feature of nodules; and (3) complete feature space determination and neural classification of nodules. Chiou et al. classifies suspected nodule areas into different anatomic structures, including rib crossing, rib-vessel crossing, end vessel, vessel cluster, rib edge, vessel, and bone (causing false positive detection). These structures, as well as a true nodule, are used as training classes to develop a neural network classifier. Note that the Chiou et al. method does not include segmentation of the lung zone into different zones or the use of that segmentation in the analytical/diagnostic processes.
The present invention for detection of abnormalities, like lung nodules, in a radiological chest image overcomes the foregoing and other problems associated with the prior art by applying a divide-and-conquer approach. Based on the knowledge of rib cage (i.e., boundary of the lung zone), the present invention divides the lung zone into different zones of similar image characteristics (of both nodule and normal anatomic structure) and conquers the problems of reducing the false positives and increasing the true positives by utilizing different digital image processing techniques and training different neural network classifiers based on the image characteristics of each zone. The lung zone is segmented into different zones, such as spine, clavicle, mediastinum, peripheral lung edge, peripheral lung central, and heart zones. For each zone, different image processing techniques are applied to enhance object-to-background contrast and to select suspected abnormality areas (SAAs). Furthermore, the present invention uses feature extraction and neural networks developed and trained specifically for each zone to finally classify the SAAs to maximize the detection of true nodules within radiological image. The findings of the potential SAAs in each zone are clustered together and used to train neural network classifiers. To avoid potential boundary problems, the zones may overlap each other. The invention develops zone-specific feature extraction algorithms for each of the zones to extract image features of the SAAs located in each particular zone. The invention trains each zone-specific classifier(s) using the SAA of that zone. Different zone-specific classifiers are trained to have different sensitivities and specificities on each zone. The invention uses SUB-Az (read xe2x80x9cSUB-A-SUB-Zxe2x80x9d) to validate different classifier performance. Some classifiers will have very high specificity (i.e., very low false-positive rate) with relative low sensitivity, while some will have very high sensitivity performance.
The present invention can be implemented in a parallel processing environment. Each zone can be processed independently of each other zone. The final output of the system is produced by a data fusion unit that optimally combines the outputs from different classifiers based on each classifier""s sensitivity and specificity performance.
An embodiment of the invention includes a system containing parallel processors that process different zones in parallel and a data fusion unit that combines the output from different classifiers. The different classifiers are trained using SAAs of the individual zones.